


























Agenda 


1. Dynamic Admission Control 
What is it? Why would you use it? 


2. OPA Gatekeeper 
Implementation Details 


3. Demonstration 
Installation and Usage 


4. Alternatives 
Standalone OPA, Kyverno, Kubewarden 
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Dynamic Admission 
Control 


https://kubernetes.io/docs/reference/access- 
authn-authz/extensible-admission-controllers/ 
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Admission Webhooks 


"... are HTTP callbacks that receive admission requests 


and do something with them.” 


— Two kinds: — Components: 
— Validating — Policy 
— Mutating — Match 
— Failure 


— Reinvocation 
— Selector 

— Namespace 

— Object 


— Rules 
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— Warnings: 


Reinvocations 
Deadlocks 
Side Effects 
Ultimacy 





— Best Practices: 


Idempotency 
Validate Mutations 


Reasonable 
Timeouts 


Equivalent Match 
Policy 


Avoid kube-system 
namespace 








Why is mutation hard? 


--smythe@google.com 


— Eventual Consistency 


— There are no 
guarantees that a 
Kubernetes resource 
will exist prior to 
another. 


SUSECON digital 


— Invocation 


— There is no guarantee 
that a mutation will 
only run once. 


Isolation — 


Operations that are = 
idempotent at one 

scope are not 

necessarily idempotent 

at every scope. 





Configuration 


Not all resources 
applied to the 
Kubernetes APIs follow 
the same standards. 








Types of Mutations 


..With simple examples 


— Convergence 
— © Happy Path 


— Regardless of how many 
applications of the mutation 
logic, the result will remain 
the same. 


— e.g. 
— Adda key value pair if it 
does not exist. 
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Oscillating Divergence 
€) Unhappy Path 


Two or more mutations work in 
opposition, causing the final 
state to oscillate. 


e.g. 
— Set container a to image 
version of b 


— Set container b to image 
version of c 


— Set container c to image 
version of a 





Unbound Divergence 
& Unhappy Path 


One or more mutations will modify 
an admission request with each 
Invocation. 


e.g. 
— Add a prefix string if it doesn't 
already exist 


— Add a different prefix string if it 
doesn't already exist 











Open Source Solution Approaches 


What are modern solutions doing to tackle this complexity? 


— Provide the user with a fully extensible system with — Provide the user with a restricted system which 
which to make mutations. restricts what type of mutations can occur. 
— Provide access to a Turing complete system. — Embed mutation validation to prove eventual 
convergence. 


— Approach taken by: 


— Kyverno — Approach taken by: 


— Kubewarden — OPA Gatekeeper 
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OPA Gatekeeper 


https://github.com/open-policy-agent/gatekeeper 
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Open Policy Agent 


Abbreviated to OPA and pronounced “oh-pa” 


Request, Event, etc. 


Decision 
(any JSON Value) 


Query 
(any JSON value) 


Data 
(JSON) 


Policy 
(Rego) 
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Policy Decoupling 


Proprietary Policy Language (Rego) 


CNCF Graduated 


Written in Go 


https://play.openpolicyagent.org/ 
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Gatekeeper 


Policy 
Template 
CRD 
Policy 
Instance 
CRD 


Audit 
results 
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apiserver 


cana PES AuthZ Admission 
Service Deploy Webhook Controller 


Config 


Admission 
Review 


Replicate 





Highly Available 





Helm Chart Installation 


Constraint and Constraint Template CRDs 


github.com/open-policy-agent/gatekeeper-library 
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Gatekeeper Mutation 


*Mutation is still experimental” 

— As of release v3.4.0 you can install it through helm by setting 
experimentalEnableMutation: true 

— Requires the deployment of two CRDs: Assign and AssignMetadata 

— Basic Conditionals: Path Tests and Value Tests 

— Does not leverage Rego 

— Logs mutations 


— Validates mutation configs with validation webhook. 
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Demonstration 


https://github.com/open-policy-agent/gatekeeper 
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~/opa 
on A k3d-k3s-default 
> 
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Alternatives 


- https://github.com/open-policy-agent/library 
- https://github.com/kyverno/kyverno 
- https://github.com/kubewarden/policy-server 
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Open Policy Agent Standalone 


Deploying OPA with a library that handles 
the mutation and injecting it as a mutating webhook. 


— Requires User Maintenance: 
— Mutating Webhook 
—  Validating Webhook 
— Open Policy Agent Configuration 


— Leverages Rego instead of CRDs 


— Not heavily promoted, shallow ecosystem 
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44 lines (39 sloc) 





1.25 KB 


package library. kubernetes.admission.mutating 


SETS SSI SSIS SSS EEE 
# PATCH rules 

= 

# Note: All patch rules 
SS SSTS 2222 


f add foo label to Dogs 

patch[patchCode] 1 
isValidReguest 
isCreateOrUpdat 
input.request.k: 
not hasLabelWVal 
patchCode = mak 


# add baz label if it hi 

patch[patchtode] { 
isValidRegquest 
isCreateOrUpdat 
input.request.k 
hasLabelValue(ii 
not hasLabelWal: 
patchtode = mak 


* add quuz label to Dog 

patch[patchCode] 1 
isValidReguest 
isCreateOrUpdat 
input.request.k 
hasLabelValue(ii 
patchtode = mak 


# Dogs get a rating 

patch[patchCode] 1 
isValidReguest 
isCreateüOrUpdat 
input.request.k 
not hasAnnatati 


patchtode = maki 


208 lines (168 sloc} 


4.69 KB 





D #4 u 


Raw Blame 


Raw Blame 


Q 2 Ù 


package library.kubernetes.admission.mutating 


io 
# Implementation of t 
* combining validatin 
zzzzzzzzczzzzzlzlzzzzzllcl 


default apiVersion - 


apiVersion = input.ap 


# missing uid default 
# this will produce a 


default response uid © 


response uid - input. 


main = ( 
"apiVersion": 
a" kind" 2 "Admi 


"response": r 


€ non-patch response 


response = x 1 


count(patch) : 
X i= {i 
"allo 
"uid" 
"stat 
1 


reason = conc 


reason != "" 


# patch response i.e. 
else = x { 
count(patch) 


mutating.md Raw 


MutatingAdmissionWebhook Example with OPA 


This is a quick example of how to use OPA as a Mutating Admission Controller in Kubernetes 1.9. You will need to run 
openpolicyagent/opa:0.6.1-dev. 


Steps 


1. Register OPA as a MutatingAdmissionWebhook 
2. Load a policy to test mutation 
3. Exercise the policy 


1. Register OPA as a MutatingAdmissionWebhook 


The steps in Kubernetes Admission Control show how to deploy OPA as a Validating Admission Controller. To deploy OPA as a 
Mutating Admission Controller, follow steps 1-3. In step 3, instead of creating a ValidatingWebhookConfiguration, create a 
MutatingWebhookConfiguration: 


kind: MutatingWebhookContiguration 
apiVersion: admissionregistration.k&s.io/vibetal 
metadata: 

name: opa-mutating-webhook 


~/opa-standaLone 
on A Local 
> 
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~/opa-standaLone 
on A k3d-k3s-default opa 
> 
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Kyverno 


Similar framework to OPA Gatekeeper 


— Installation managed by Helm chart 
— Leverages CRDs for Mutation and Validation 


— Two methods of mutation: 
— JSONPatch 
— Strategic Merge Patch 


— Allows for Mutate Rule Ordering 
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Fe realshuting Fix log message (#1779) 





crds 


templates 


Chart.yaml 
README.md 


values.yaml 


README.md 


Kyverno 


Kyverno is a Kuberr 


e Manage policie 
e Validate, muta! 
e Select resource 
e View policy en 


e Scan existing r 


Access the complet 
TL;DR 


## Add the Kyver 
$ helm repo add 


## Install the K 
$ helm install k 


Introduction 
This chart bootstra| 
Installing the 


Add the Kyverno H 


43 lines (43 sloc) 


1.25 KB 


apiVersion: kyverno.io/v1 


kind: ClusterPolicy 
metadata: 
names inject-sidecar 
annotations: 
policies.kyverno.lo, 
policies.kyverno.io, 
policies.kyverno.io, 
Sample policy tha 
spec: 
background: false 
rules: 
- name: inject-sidecal 
match: 
resources: 
kinds: 
- Deployment 
mutate: 
patchstrategicMer;| 
spec: 
template: 
metadata: 
annotatiol 
(vault.! 
spec: 
container: 
- name: vi 
image: ' 
imagePu. 
volumeMi 
- mount! 
name: 
initConta: 
- name: vi 
image: ' 
imagePu. 
volumeMi 
- mountl 
name: 


volumes: 





Y f3cald7 2 days ago 9 History 
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Raw Blame 


RFC 6902 JSONPatch = 


A JSON Patch, implemented as a mutation method called patchesIson6982 , provides a precise way to mutate resources and supports the following operations (in the op field): 


e add 
e replace 


* remove 
With Kyverno, the add and replace have the same behavior (i.e., both operations will add or replace the target element). 


The patchesJson69e2 method can be useful when a specific mutation is needed which cannot be performed by patchesstrategicMerge . For example, when needing to mutate a 
specific object within an array, the index can be specified as part of a patchesJsonese2 mutation rule. 


One distinction between this and other mutation methods is that patchesJsonese2 does not support the use of conditional anchors. Use preconditions instead. Also, mutations 
using patchesJson69e2 to Pods directly is not supported as the rules are not converted to higher-level controllers such as Deployments and StatefulSets through the use of the 
auto-gen feature. Therefore, when writing such mutation rules for Pods, it may be necessary to create multiple rules to cover all relevant Pod controllers. 


This patch policy adds, or replaces, entries in a ConfigMap with the name config-game in any namespace. 


apiVersion: kyverno.io/v1 
kind: ClusterPolicy 
metadata: 


name: policy-patch-cm 


spec: 
rules: 
- name: pCMI 
match: 
resources: 


name: "config-game" 
kinds: 
- ConfigMap 
mutate: 
patchesJson6902: |- 
- path: "/data/ship.properties" 
op: add 


AU Int | 


~/kyverno 
on A local 
> 
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~/kyverno 
on A k3d-k3s-default 
> 
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Kubewarden 


Deploys a controller that generates 
a policy server from which WASM policies are served. 


— Can be installed with Helm chart 
— Allows you to write in any language that can be compiled into WASM 
— SDKs for Golang and Rust 


— Policies are binaries that can be served from the following locations: 
— Local Filesystem 
— HTTP Server 
— OCI Compliant Registry 
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Install 


The Kubewarden stack o 
Kubewarden Policies 


helm repo add kubewz 


helm install --names Enforcing policies is | AAA "m 
You can declare as n extern crate wapc guest as guest; 
This will install FE operation that can D è > use guest: :prelude::*; 
will register the Admiss The ClusterAdmissi ^4 use k8s openapi::api::core::vl as apicore; 
De deployed inside of a policies are defined. extern crate kubewarden policy sdk as kubewarden; 


use kubewarden::{request::ValidationRequest, validate settings}: 


The default configuratit . . m 
apiVersion: polic 


options are documente kind: ClusterAdmi; © "5 settings; 
TOU 18 use settings::Settings: 


metadata: 
The Kubewarden Policy name: psp-capab — 
spec: 1: pub extern "C" fn wapc init() 1 
module: regi str 14 register function("validate", validate); 
resources; 15 register function("validate settings", validate_settings::<Settings>); 
- pods 16 } 
operations: M 
— CREATE li fn validate(payload: £[u8]) -> CallResult { 
— UPDATE let validation request: ValidationRequest<Settings> = ValidationRequest::new(payload)?; 
mutating: true | | | 
: // TODO: you can unmarshal any Kubernetes API type you are interested in 
settings: 2: match serde_json::from_value: :<apicore: :Pod>(validation_request.request.object) { 
allowed_capab J: Ok(pod) => 4 
— CHOWN 24 // TODO: your logic goes here 
requi red d rop 2! if pod.metadata.name == Some("invalid-pod-name".to string()) Y 
— MET ADMIN 2f kubewarden::reject reguest( 
Some(format!("pod name {:2} is not accepted", pod.metadata.name)), 
Mone, 
) 
TR El y else { 
SUSECON dig ital? 37 kubewarden::accept reguest() 


~/kubewarden 
on A local kubewarden 
> | 
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~/kubewarden 
on A k3d-k3s-default 
) 
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